- What is the problem?
When a website undergoes code revisions, breaking changes can be introduced.
- Why is it a problem?
Clients don’t like surprise breaking changes. We can manually test for lots of different visual page breakages, but it chews up billable hours that could be better spent on features.
- How do you solve the problem today?
We manually test multiple pages to see if the changes to code have also changed the layout in unexpected or breaking ways.
- How would you ideally solve the problem?
We would love to be able to run a VRT like is done using Autopilot. It could be the same ones we already have defined in Autopilot. If Autopilot isn’t set up, or is not allowed to be on a site, we’d need a separate mechanism for designating which URLs to test. In that case, perhaps they could be part of a list in the pantheon.yml file or in the wp-config.php.
- How big is the problem (business impact, frequency of impact, who is impacted)
The business impact varies depending on the severity of a missed breaking change to a layout. If it’s a minor problem on an About Us page, that’s different than if the whole home page loses styling. Again, manual tests or using Diffy.website are options, but manual tests are subject to human error (if they get done at all) and paying for yet another visual regression testing service on top of what we pay for Pantheon is a non-starter in most cases. Especially when Diffy and others don’t work the same as Autopilot’s VRT engine.
